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EXECUTIVE  SUMMARY 


The  Direction  Finder  (DF)  network  is  maintained  and  operated  by  the  Federal 
Aviation  Administration  (FAA)  as  a  position  location  service  for  aircraft.  The 
present  network  is  made  up  of  a  mix  of  tube- type  and  solid-state  equipment.  The 
tube -type  equipment  will  be  replaced  by  new  solid-state  equipment,  the  FA- 10121, 
which  incorporates  operator  task  automation,  remote  maintenance  monitoring,  and 
control  and  certification  capabilities. 

Operational  Test  and  Evaluation  (OT&E)  Integration  and  OT&E  Operational  testing  was 
conducted  on  the  Very  High  Frequency  (VHF)  Direction  Finder  (VDF)  at  the  Green  Bay, 
WT ,  Automated  Flight  Service  Station  (AFSS),  from  March  29  through  April  9,  1993, 
using  software  release  4.05,  and  June  21  through  June  24,  1993,  using  VDF  software 
release  4.07.  Testing  was  categorized  into  the  areas  of  integration  and 
maintenance,  display  and  keyboard  functions,  and  operational  flight  testing. 

Flight  testing  utilized  the  FAA  Technical  Center  Aero  Commander  680E  aircraft. 
Results  were  generally  satisfactory  in  all  test  areas.  However,  some  deficiencies 
exist  in  the  maintenance  and  operational  areas  and  deviations  from  volumes  I  and 
III  of  the  National  Airspace  System  (NAS)  System  Specification,  NAS -SS- 1000,  were 
also  noted. 

It  is  concluded  that  operationally,  the  system  is  acceptable  for  deplojrment. 
Institution  of  a  plan  of  action,  to  address  the  observed  deficiencies,  is 
recommended. 
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UCTION. 


1.  I 


1.1  PURPOSE. 

This  report  provides  the  formal  results  of  the  Operational  Test  and  Evaluation 
(OT&E)  Integration  and  OT&E  Operational  testing  that  was  conducted  on  the  new 
FA- 10121  Very  High  Frequency  (VHF)  Direction  Finder  (VDF)  at  the  Green  Bay,  WI , 
Automated  Flight  Service  Station  (AFSS).  It  covers  testing  accomplished  to  verify 
that  the  system  meets  the  requirements  identified  in  the  Direction  Finder  (DF)  Test 
Verification  Requirements  Traceability  Matrix  (TVRTM)  from  the  0T6eE  Integration  and 
OT&E  Operational  Test  Plan. 

1.2  OVERVIEW  OF  REPORT. 

The  DF  has  been  the  subject  of  OT&E  testing  activities  performed  at  the  Federal 
Aviation  Administration  (FAA)  Technical  Center,  Atlantic  City  International 
Airport,  NJ,  and  at  the  FAA  AFSS  in  Green  Bay,  WI,  at  various  times  between 
January  1987  and  June  1993.  This  report  covers  recent  testing  at  Green  Bay  AFSS 
in  May  and  Jiuie  1993,  which  emphasized  operational  performance  and  functionality, 
and  also  served  to  verify  those  enhancements  made  under  contract  modification  37 
to  DTFA01-85-01003 .  Testing  was  divided  into  three  categories;  (1)  integration 
and  maintenance,  (2)  keyboard  and  display,  and  (3)  operational  flight  testing. 

This  report  covers  the  methodology  used,  results  of  tests  compared  to  test 
requirements ,  and  provides  a  rationale  for  the  recommendation  for  deployment 
made  in  section  7 . 


2  ■  DOCUMENTS . 


FAA-PD-420-02a  Purchase  Description,  Direction  Finder  Replacement  and 
Modernization,  January  1993. 


FAA-STD-024a  Preparation  of  Test  and  Evaluation  Documentation, 
August  17,  1987. 

FAA  Order  FAA  NAS  Test  and  Evaluation  Policy,  October  1992. 

1810.4b 


NAS -SS- 1000  Functional  and  Performance  Requirements  for  the  NAS 

Maintenance  and  Operations  Support  Element,  Volumes  I 
,  and  III,  December  1986,  with  changes  to  March  and  May  1992, 

respectively, 

DOT/FAA/CT-  FA-9964  Installed  Accuracy  Evaluation,  September  1991. 

TN91/25 


FAA  MTP 


VHF  Direction  Finding  (VDF)  System,  Master  Test  Plan, 
Revised  September  1992. 
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FAA  OT&E  Plan  VHF  Direction  Finder  Operational  Test  and  Evaluation 
(OT&E)  Integration  and  OT&E  Operational  Test  Plan, 
March  3,  1993. 


FAA  OT&E  Test  VHF  Direction  Finder  Operational  Test  and  Evaluation 

Procedures  (OT&E)  Integration  and  OT&E  Operational  Test 
Procedures,  April  5,  1993. 

Quick  Look  VHF  Direction  Finder  Operational  Test  and  Evaluation 

Report  (OT&E)  Integration  and  OT&E  Operational  Test 
Quick  Look  Report,  July  12,  1993. 


FAA-D-8h(2)  VDF  Air  Traffic  Specialist,  System  User  Guide, 

April  29,  1992. 


TI  6530.10  VHF  Direction  Finder,  Receiver/Processor  Group, 

July  30,  1991. 

TI  6530.11  DF  Remote  Maintenance,  Monitor  and  Control  (RMMC) /Information 

Display  and  Control  Unit  (IDCU)  Subsystems,  November  16,  1992. 


FAA  OTCiE  Test  VHF  Direction  Finder  (VDF)  Operational  Test  and  Evaluation 
(OT&E)  Integration  and  OT&E  Operational  Test  Logs  and  Data, 
DOT/FAA/CT-TN9V13 . 


3.,  .SYSTEM  DESCRIPTION. 


WIgSIQy  REVIEW-. 

The  DF  network  is  maintained  and  operated  by  the  FAA  as  a  position  location  service 
for  aircraft.  An  expanded  network  operated  by  Flight  Service  Station/Automated 
Flight  Service  Station  (FSS/AFSS)  sites  is  necessary  to  provide  rapid  position 
orientation  for  aircraft  that  are  uncertain  of  their  position  or  in  distress,  and 
to  provide  position  reference  information  to  aircraft  upon  request. 

The  present  network  is  made  up  of  a  mix  of  tube- type  and  solid-state  equipment. 

The  new  equipment  nomenclature,  FA- 10121,  includes  new  DF  antennas  for  some  antenna 
sites.  At  other  sites,  the  existing  FA-9964  DF  antennas  are  retained  and  are 
interfaced  with  the  FA- 10121  operating  system.  The  new  system  incorporates  remote 
maintenance  monitoring  (RMM)  Including  control  and  certification  capabilities , 
between  the  DF  site  antennas  and  the  AFSS  where  the  operating  position  is  located. 

The  new  system  incorporates  DF  bearing  information  from  up  to  24  remotely  located 
DF  antennas  (any  mix  of  FA- 10121  or  FA-9964  antennas),  a  site  specific  geographic 
map,  and  an  aeronautical  data  base,  on  a  graphic  display  terminal.  These  features 
enhance  the  specialist's  ability  to  locate  aircraft  in  a  timely  manner  by 
eliminating  the  manual  plotting  of  an  aircraft's  position  and  by  retrieval  of 
reference  information. 
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The  VHF  DF  system  operates  over  a  frequency  range  of  118.000  to  136.975  megahertz 
(MHz)  and  provides  760  channels  spaced  every  25  kilohertz  (kHz)  with  10  preset 
channels.  The  system  is  capable  of  receiving  an  aircraft  transmission  on  one  or 
more  DF  antenna  sites  and  providing  the  audio  to  the  operator.  It  also  presents  to 
the  operator,  in  graphical  form,  position  information  on  the  aircraft  relative  to 
other  points  of  interest.  When  triangulation  is  possible  using  two  or  more  DFs, 
the  aircraft  location  is  automatically  displayed  on  the  operator's  display,  the 
Information  and  Display  Control  Unit  (IDCU).  Upon  request,  the  heading  and 
distance  to  the  six  nearest  airports  can  be  displayed. 

3.2  TEST  SYSTEM  CONFIGURATION. 

The  tested  system  as  installed  at  the  Green  Bay,  WI,  AFSS  was  configured  with  three 
FA- 10121  antennas  and  four  FA- 9964  antennas.  The  seven  antennas,  with  their 
Receiver/Processor  Groups,  were  at  various  locations  throughout  Wisconsin.  See 
figure  3.2-1  for  the  locations.  Information  from  the  antennas  is  carried  by 
dedicated  telephone  lines  and  received  by  seven  modems  in  the  Remote  Maintenance 
Monitor  Computer  (RMMC)  subsystem  rack  located  in  the  AFSS.  An  Input/Output 
Terminal  Type  2  (TOT-2),  and  a  printer  are  attached  to  the  RMMC  for  maintenance 
actions.  Two  IDCUs,  which  constitute  the  AFSS  Specialists'  work  station  were 
installed,  one  in  the  operations  room  and  another  in  an  adjacent  training  room. 

See  figure  3.2-2  for  a  block  diagram  of  the  equipment  as  installed  at  Green  Bay 
AFSS, 

Software  utilized  in  the  Very  High  Frequency  Direction  Finder  (VDF)  Preliminary 
0T6iE  conducted  from  March  29  through  April  9,  1993,  was  version  4.05.  Computer 
printouts  showed  version  4.07,  June  14,  1993,  in  use  between  June  18  and  24,  1993, 
for  OT&E,  and  then  version  4.07,  June  18,  1993,  in  use  on  June  25,  1993,  A  site 
adaptation  and  data  base  software  file  unique  to  Green  Bay  AFSS  Flight  Plan  Area 
was  utilized  on  both  occasions. 

3 . 3  INTERFACES . 

3.3.1  Actual ■ 

Actual  interfaces  utilized  during  testing  consisted  of  those  internal  to  DF  system 
components,  from  the  DF  to  aircraft  radios,  and  the  computer  human  interface  (CHI) 
between  the  DF  and  the  AFSS  specialists.  This  included  RMM  and  Certification 
capabilities  available  between  the  remotely  located  DF  antennas  and  the  AFSS. 

3.3.2  Simulated. 

During  testing  conducted  at  the  FAA  Technical  Center  in  December  1992,  21  simulated 
DF  antennas  were  used  in  addition  to  3  actual  antennas ,  to  provide  a  maximum 
loading  of  24  antennas.  At  Green  Bay,  no  simulated  Interfaces  were  required  or 
utilized.  All  actual  interfaces  required,  except  as  noted  below  in  paragraph 
3.3.3,  were  installed  and  tested. 

3.3.3  Deferred. 

The  proposed  interface  from  the  DF  system  to  the  National  Airspace  System  (NAS) 
Maintenance  Processor  Subsystem  (MPS)  was  unavailable  and  testing  of  such  has  been 
deferred. 
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FIGURE  3.2-1,  DF  ANTENNA  LOCATIONS 


FIGURE  3.2-2.  DF  EQUIPMENT  CONFIGURATION 
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4,  TEST  DESCRIPTION. 


The  OT&E  Integration  and  OT&E  Operational  testing  was  conducted  at  the  FAA 
Technical  Center  and  Green  Bay,  WI,  AFSS.  Technical  testing  of  the  DF  antenna  and 
receiver/processor  group  components  for  bearing  accuracy  was  performed  in  January 
through  March  1987,  and  was  not  repeated  in  this  series  of  tests.  However,  the 
flight  testing  portion  of  this  OT&E,  utilizing  the  complete  FA- 10121  DF  system,  did 
encompass  accuracy  of  position  measurements.  This  test  included  a  comparison  of 
the  FA- 10121  antenna  subsystem  and  the  FA- 9964  antenna  subsystem  within  the  overall 
FA- 10121  system  from  an  operational  viewpoint.  Initial  OT&E  Integration  and  OT&E 
Operational  testing  took  place  at  the  FAA  Technical  Center  utilizing  3  actual  and 
21  simulated  DF  antenna  sites.  Subsequently,  the  DF  was  tested  at  the  Green  Bay, 

WI ,  AFSS.  At  Green  Bay,  an  operational  network  of  seven  DF  antenna  sites  was 
utilized  to  verify  interfaces  and  operational  performance.  The  FAA  Technical 
Center  Aero  Commander  680E  aircraft  was  used  for  airborne  testing. 

4.1  SCHEDULE  AND  LOCATION. 

The  OT&E  Integration  and  OT&E  Operational  testing  activity  was  conducted  at  three 
locations  on  several  dates .  Testing  took  place  at  the  FAA  Technical  Center  and  the 
Millville,  NJ ,  AFSS  in  1988.  Additional  testing  was  performed  at  the  FAA 
Technical  Center  in  December  of  1989  and  1992.  Testing  was  also  performed  at  Green 
Bay  AFSS  during  February  1990,  and  February,  April,  and  June  of  1993.  In  each 
case,  testing  locations  included  the  IDCU  work  station  area,  the  RMMC,  and  the 
local  or  remotely  located  DF  antennas.  The  VDF  has  undergone  modifications  and 
enhancements  over  Che  years  including  several  made  under  contract  modification  37 
to  DTFA01-85-y01003 .  The  DF  system  tested  in  1993  encompasses  those  enhancements 
and  it  the  subject  of  this  report  on  the  OT&E  performed  at  Green  Bay  AFSS  during 
April  and  June  of  1993. 

4.2  PARTICIPANTS. 


ACW-300A; 


ACN-710: 

ATR-130: 


Conducted  the  OT&E.  Joseph  P.  Pino,  Associate  Program 
Manager  for  Test,  (APMT) ;  John  Dyson,  Test  Director; 
Robert  Bernheisel,  Norman  Beauregard,  and  Sam  Barto, 

Test  Technicians;  and  Ron  Lockhart,  Flight  Test  Engineer. 

Provided  flight  test  aircraft  and  pilot.  Theos  McKinney. 

Participated  as  a  coordinator  and  monitor  of  OT&E. 

Leonard  Hopkins,  Gib  Shade. 


ATZ-120 
ATM- 110 
ATP- 110 
ATR-200 
AOS -240 


Participated  as  a  monitor.  Charlie  Parks. 

Participated  as  a  monitor.  Bill  Fish. 

Participated  as  observer.  Gene  Higgins 

Observer  of  OT&E  operational  aspects.  Kevin  Harrington. 

Participated  as  a  monitor.  Ed  Lugo. 
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AMA-571: 


Observer  in  the  area  of  training  education.  Rick  Akers. 
Attended  as  observers;  Greg  Rugila,  Bill  Swart,  Dana  Dias. 


ANN- 600: 

Green  Bay  AFSS:  Participated  as  a  host  facility,  and  provided  DF 
operators ,  and  maintenance  technicians . 

Hughes  SIX:  Development  contractor  personnel  that  participated 

as  observers  and  provided  technical  support; 

John  Morris,  Rico  Honey,  and  others. 

4.3  TEST  EQUIPMENT. 

4.3.1  Standard. 

Standard  FAA  test  equipment  was  used  during  testing  of  the  DF  antenna  subsystems, 
the  RMMC,  and  modems  which  included:  a  VOM,  signal  generator,  and  a  frequency 
counter . 

Ll-I.  ? _ Special  Test  Equipment. 

A  COMB  generator,  supplied  as  a  part  of  the  DF  system  was  utilized  in  DF  antenna 
ground  check  procedures.  An  Input/Output  Terminal  T3rpe  3  (IOT-3),  which  is  a 
portable  personal  computer  with  maintenance  software  was  also  used  at  ancenna 
sites.  The  IOT-2,  part  of  the  DF  RMMC  system,  was  used  during  RMMC  testing. 

The  FAA  Technical  Center  Aero  Commander  680E,  call  sign  November  50  (N50) ,  with 
standard  Instxxunent  Flight  Rules  (IFR)  certified  equipment  was  utilized  for  the 
operational  flight  testing.  A  hand-held  Garmin  AVD-32A  Global  Positioning  System 
(GPS)  unit  was  also  used  on  board  N50  to  establish  actual  aircraft  position  during 
the  flight  scenarios.  An  additional  IFR  certified  light  aircraft,  a  Maul  XMT-180, 
call  sign  N9228L,  with  two  VHF  radios,  dual  VHF  Omnidirectional  Radio  Range  (VOR) , 
Distance  Measuring  Equipment  (DME) ,  and  Automatic  Direction  Finder  (ADF)  was  used 
for  the  "Two-Aircraft"  Flight  Scenarios  during  preliminary  OT&E  testing. 

4.3.3  Simulators. 

Contractor  provided  DF  antenna  simulators  were  utilized  during  testing  at  the 
FAA  Technical  Center  in  December  1992 ,  to  simulate  21  antennas  in  addition  to  the 
3  actual  antennas  to  provide  a  maximum  loading  of  24  antennas . 

4.4  TEST  OBJECTIVES. 

Test  objectives  were  derived  from  the  Test  Verification  Requirements  Traceability 
Matrix  (TVRTM)  contained  in  the  OT&E  Test  Plan,  which  are  derived  from  volumes  I 
and  III  of  NAS -SS- 1000,  as  specified  by  the  FAA  VDF  Master  Test  Plan.  Special 
emphasis  was  placed  on  verifying  that  the  required  corrections  and  desired 
enhancements  identified  during  DF  testing  in  February  1990  have  been  Incorporated 
into  the  DF  system.  Additional  objectives  were  derived  from  VDF  contract 
modification  37  and  AFSS  specialists  requirements.  All  of  these  were  carried  into 
the  OT&E  Test  Plan  TVRTM  and  are  summarized  herein. 
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4.4.1  Integration  and  Maintenance  Area. 


Verification  of  basic  system  startup  and  operation.  This  included;  system  power 
up  sequence,  initialization,  certification,  radio  reception,  diagnostics,  and 
selected  maintenance  procedures.  The  verification  also  checked  that  the  existing 
FA- 9964  and  new  FA- 10121  DF  antennas  operate  properly  with  the  new  DF  system. 

4.4.2  Display  and  Keyboard  Area. 

Verification  of  the  operator  display  and  keyboard  functions  including:  maps, 
scales,  symbology,  color  codes,  label  functions,  data  base  access,  and  position 
calculation  and  display. 

4.4.3  Operational  Flight  Test  Area. 

Verification  of  DF  system  operational  performance  including  AFSS  operator, 
navigational  aids,  and  aircrew  elements  of  the  total  system.  Areas  addressed 
included:  DF  operator  determination  of  aircraft  position  via  multiple  DF  bearings 
(via  DF  and  a  Navigational  Aid  (NAVAID)),  display  of  geographical  features, 
airports ,  NAVAIDs ,  Military  Operations  Areas  (MOA) ,  and  creation  and  display  of 
reference  lines.  Also,  verification  that  the  operator  can  utilize  the  position 
data  to:  direct  the  aircraft  to  an  airport,  provide  pertinent  information  to  the 
pilot  about  aircraft  position,  landmarks,  obstructions,  and  NAVAIDS.  Additionally, 
verification  that  the  DF  operator  can  direct  a  pilot  to  an  airport  in  a  position 
from  which  a  safe  landing  can  be  made.  Emphasis  was  also  given  to  noting  any 
display  inconsistencies  between  the  two  installed  IDCUs. 

4 . 5  TESTING  CATEGORIES . 

The  following  test  categories  are  as  listed  in  the  0T6iE  Test  Plan  and  Test 
Procedures . 

4.5.1  Integration  and  Maintenance  Functions  (Category  11. 

This  category  covered  the  following  areas: 

a.  Physical  compatibility 

b .  Turn  on  and  Power  up  sequence 

c.  Initialization 

d.  Certification 

e .  Diagnostics 

f.  RMM  of  antennas  to  DF  system 

g.  RMMC  switchover 

h.  IOT-2  and  IOT-3 

i.  COMB  generator  applications 
J .  Audio  interface 

k.  Key  Line  interface 

l.  Strobe  Inhibit 
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— Display  and  Keyboard  Functions  (Category  2). 

This  category  covered  the  following  functions: 

a.  DF  Receiver  Site  Locations 

b.  VHP  Transmission  Reception 

c.  Bearing  Strobe 

d.  Navigational  Aids 

e .  Airport  Locations 

f.  Obstruction  Locations 

g.  Data  Base  Updates 

h.  Labels 

i .  Overwrite 

j .  Frequency  Selection  (manual  and  automatic) 

_ Operational  Flight  Testing  (Category  3). 

This  category  covered  the  following  functions: 

a.  Aircraft  position  determination  by:  DF/VOR,  DF/NDB,  2  DF,  3  DF, 
DF/Reference  line,  and  Time/Distance  calculation 

b.  Guidance  of  aircraft  to:  airports,  landmarks,  NAVAIDS 

c.  DF  station  passage  and  DF  approaches 

d.  Hap  symbology 

e.  Two 'Aircraft  scenario 

f.  Data  base  access 

g.  Emergency  aircraft  mode 

4.6  DATA  COLLECTION  AND  ANALYSIS  METHOD. 

Data  points  were  collected  throughout  the  test  activity  utilizing  the  prepared 
forms  and  logs  contained  in  the  OT&E  Test  Procedures. 

4.^.1 _ Integration  and  Maintenance  Functions  (Category  1). 

Normal  setup  and  initialization  procedures  were  followed  according  to  the  DF 
manufacturers  handbooks  and  the  test  procedures.  The  IDCU,  IOT-2,  IOT-3,  and  COMB 
generator  were  exercised.  Test  monitors  recorded  the  data  generated  and  displayed 
by  the  lOTs  and  IDCUs  in  response  to  test  inputs,  as  well  as  narrative  comments  on 
system  operation. 

The  collected  data  points  were  reviewed  and  compared  to  system  specifications  and 
handbooks  to  verify  proper  operation.  The  OT&E  test  logs  are  contained  in  sections 
1  and  2  of  VHF  Direction  Finder  (VDF)  Operational  Test  and  Evaluation  (OT&E) 
Integration  and  OT&E  Operational  Test  Logs  and  Data. 
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^.^.2 _ Display  and  Keyboard  Functions  (Category  2). 


At  the  DF  IDCU,  an  operator  exercised  keyboard  and  trackball  functions  according  to 
the  operators  manual  and  test  procedures.  The  IDCU  display  and  readouts  were 
observed  for  the  appropriate  responses  and  consistency  with  FAA-D-8h(2),  the  DF  Air 
Traffic  Specialist  System  User  Guide.  Test  monitors  logged  results  consisting  of 
specific  IDCU  responses  and  readouts,  and  general  narrative  comments. 

The  collected  data  points  were  reviewed  and  compared  to  system  specifications, 
handbooks,  and  known  data  bases  to  verify  proper  display  and  accuracy  of  presented 
data.  The  OT&E  test  logs  are  contained  in  sections  1  and  2  of  VHF  Direction  Finder 
(VDF)  Operational  Test  and  Evaluation  (OT&E)  Integration  and  OT&E  Operational  Test 
Logs  and  Data. 

4.6.3  Operational  Flight  Testing  (Category  3). 

Category  3  testing  consisted  of  exercising  operational  flight  scenarios.  A  Green 
Bay  AFSS  specialist  operated  the  DF  system  from  the  IDCU  as  the  FAA  Technical 
Center  Aero  Commander  aircraft  proceeded  through  each  of  the  flight  scenarios.  The 
scenarios  provided  situations  requiring  the  exercising  of  the  various  DF 
operational  capabilities.  Data  points  were  copied  by  test  monitors  directly  from 
the  IDCUs  and  narrative  comments  on  system  operation  were  made  on  the  test  forms. 
The  airborne  test  engineer  copied  aircraft  position  data  directly  from  aircraft 
instrvunentatlon  Including  VOR,  DME,  ADF,  LORAN  C,  and  the  hand-held  GPS  unit 
carried  on  board.  Visual  observations  from  the  aircraft  were  also  recorded. 
Airborne  scenarios  and  position  data  are  contained  in  sections  3  and  4  of  VHF 
Direction  Finder  (VDF)  Operational  Test  and  Evaluation  (OT&E)  Integration  and  OT&E 
Operational  Test  Logs  and  Data.  Comparisons  were  made  between;  IDCU  readouts  and 
displayed  locations  against  the  known  data  base,  and  of  aircraft  position  reported 
by  VOR  Radial/DME ,  GPS ,  LORAN  C ,  or  visual  means . 

Additionally,  some  specific  flight  testing  was  performed  utilizing  orbits  flown 
around  the  Rhinelander  and  Stevens  Point  FA- 9964  antennas;  and  the  Marquette 
and  the  Green  Bay  FA- 10121  antennas.  This  provided  data  to  assess  overall 
system  accuracy  associated  with  the  two  different  antenna  types ,  and  also  to 
assess  consistency  of  bearing  displays  between  the  two  IDCUs .  Orbits  were  flown 
at  Green  Bay  and  Stevens  Point  at  5,  10,  and  40  nautical  miles  (nmi) ,  at  Marquette 
at  10  nmi,  and  at  Rhinelander  at  10  and  20  nmi.  Data  points  were  collected  on 
board  the  aircraft,  and  also  logged  at  each  IDCU. 

The  GPS  location  was  translated  to  distance  and  azimuth  from  DF  site  in  order  to 
make  the  azimuth  error  analyses.  The  translated  data  is  combined  with  data 
collected  from  the  IDCUs  in  section  5  of  VHF  Direction  Finder  (VDF)  Operational 
Test  and  Evaluation  (OT&E)  Integration  and  OT&E  Operational  Test  Logs  and  Data. 
Judgements  were  drawn  from  debriefings  and  narrative  comments  on  data  sheets  and 
logs  concerning  system  performance. 
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5.  RESULTS  AND  DISCUSSION. 


5.1  INTEGRATION  AND  MAINTENANCE  FUNCTIONS  (CATEGORY  1). 

Verification  of  the  maintainability  aspects  of  the  system  and  its  integration  into 
the  NAS  were  the  objectives  of  this  test  series.  Discrepancies  were  noted  in  some 
test  areas  and  are  summarized  herein. 

Examination  of  the  6530.11/a  Technical  Instruction  (Tl)  manual  revealed  that  it  had 
not  yet  been  revised  to  reflect  the  current  VDF  system  configuration.  The 
commercial-off-the-shelf  (COTS)  manuals  were  sampled  and  omissions  were  noted  about 
the  FA- 9964  type  COTS  modems.  Details  of  the  modified  COTS  equipment  were  not 
contained  in  the  manufacturer's  u  umentation. 

Surge  protection  for  the  RMMCs  and  IDCUs  according  to  the  requirements  of  the 
Purchase  Description  (PD),  FAA-PD-420-02a,  section  1-3.6.12  has  not  been  provided 
by  the  contractor.  Grounding  of  the  VDF  equipment  racks  is  considered  paramount 
for  reliable  and  safe  system  operation  and  was  found  to  be  deficient  during 
preliminary  OT&E  testing  in  April  1993.  During  the  June  1993  OT&E  test,  an 
Inspection  of  the  ground  changes  made  on  the  equipment  racks  showed  acceptable 
grounding . 

Verification  of  the  actual  receiver  squelch  threshold  versus  that  selected  by  an 
operator  at  IOT-2  demonstrated  that  the  squelch  setting  was  within  3  decibel  (db) 
of  the  selected  -95  decibels  above  1  milliwatt  (dBm)  which  is  within  the 
requirements  of  the  PD.  Squelch  break  occurred  at  -117  dBm  when  -140  dBm  was 
selected  at  IOT-2.  Operation  limits  beyond  -97  dBm  are  not  specified  in  the  PD. 
From  an  operational  standpoint,  the  user  must  be  aware  that  squelch  selection  is 
neither  accurate  nor  linear  below  -97  dBm. 

It  could  not  be  initially  verified  via  documentation  nor  nondestructive  means 
whether  the  Mitsubishi  color  display  monitors  being  used  for  the  IDCU  indicators 
meet  the  implosion  protection  requirement  of  the  PD.  Post-OT&E  investigation, 
however,  revealed  that  the  displays  were  implosion  protected. 

Airway  Facilities  (AF)  technicians  demonstrated  a  satisfactory  ability  to  operate 
the  system  and  perform  limited  problem  diagnosis.  It  appeared  that  the  technicians 
had  become  more  proficient  since  the  preliminary  OT&E,  however,  uncertainty  exists 
in  evaluating  the  actual  level  of  proficiency  being  attained  through  the  current 
training  course  structure. 

Induced  failures  of  primary  RMMCs  during  actual  system  operation  resulted  in 
acceptable  switchover  to  the  backup  units.  As  expected,  an  audible  alarm  and  a 
communication  failure  message  (COMM)  were  presented  at  the  IDCU.  However,  an  alarm 
delay  period  of  approximately  20  seconds  was  noted  during  which  time  the  operator 
cannot  access  the  antenna  sites  or  receive  bearing  updates  from  them.  This  alarm 
period  is  beyond  the  1- second  status  response  time  specified  in  paragraph 
3. 2. 1.3. 7. 2. 5  of  NAS-SS-1000,  volume  III. 

Note:  NAS  Change  Proposal  16136  is  currently  in  coordination.  DF 
performance  is  expected  to  meet  the  tailored  NAS  requirement. 
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Development  and  presentation  of  operator  selected  diagnostic  test  and  facility  data 
by  the  VDF  was  observed  to  require  an  average  time  of  4.5  minutes.  This  period  is 
beyond  the  2 -minute  average  time  specified  for  this  function  in  paragraph 
3.2.1.2.9(c)  of  NAS-SS-1000,  volume  I. 

During  diagnostic  testing  from  the  IOT-2  maintenance  console,  reports  from  FA-10121 
sites  would  sometimes  indicate  failures  at  the  antenna  site(s).  However,  upon 
repeating  the  diagnostic  test  without  corrective  action,  the  failures  would 
sometimes  disappear. 

Observations  during  the  OT&E  concerning  maintenance  has  raised  the  specter  of 
vmcertainty  as  to  whether  the  4200  hour  mean- time -between- failure  (MTBF) 
requirement  is  actually  demonstrable  in  the  field.  Section  6  of  VHF  Direction 
Finder  (VDF)  Operational  Test  and  Evaluation  (OT&E)  Integration  and  OT&E 
Operational  Test  Logs  and  Data  contains  miscellaneous  data  on  equipment  failures 
and  repairs  that  occurred  during  the  6  months  of  intermittent  testing  conducted  at 
the  Green  Bay  AFSS. 

5.2  DISPLAY  AND  KEYBOARD  FUNCTIONS  (CATEGORY  2). 

Verification  of  the  IDCU  and  IOT-2  functions  revealed  no  major  problems.  Some 
discrepancies  were  noted  and  are  highlighted  herein. 

The  IDCU  keyboard  functions  worked  as  documented  on  all  features  available  in  the 
Green  Bay  AFSS  display.  A  super  site  was  created  to  test  some  features  not  in  the 
Green  Bay  AFFS;  keyboard  operation  on  the  super  site  was  also  correct.  Minor 
discrepancies  were  noted  in  the  help  text  of  several  IDCU  functions.  There  is  no 
operational  impact  to  these  text  discrepancies  though  this  demonstrates  some  lack 
of  refinement. 

Checkout  of  the  IDCU  map  revealed  some  inconsistencies,  but  no  major  problems  were 
discernible.  This  test  exercise  did  highlight  and  reinforce  the  importance  of 
verifying  the  correctness  of  the  maps  that  will  be  supplied  to  field  facilities  and 
the  requirement  for  a  viable  mechanism  for  updating  these  maps. 

Verification  of  the  security  slgn-on  and  alarm  control  functions  revealed  no  major 
operational  deficiencies.  Checkout  of  IOT-2  FSS  facility  control,  monitor, 
maintenance,  and  system  management  menus  revealed  minor  inconsistencies. 

The  IDCU  display  response  time,  when  the  calculated  position  of  a  VHF  radio 
transmission  source  is  displayed,  was  noted  as  6  to  8  seconds.  Paragraph 
3. 2. 1.3. 7. 2. 6  of  NAS-SS-1000,  volume  III,  sets  this  time  at  a  maximum  of 
3  seconds. 

Note:  NAS  Change  Proposal  16136  is  currently  in  coordination.  Average  DF  time 
to  position  presentation  is  within  the  limits  proposed  for  the  tailored  NAS 
requirement . 
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5.3  OPERATIONAL  FLIGHT  TESTING  (CATEGORY  3). 


All  five  flight  scenarios  contained  in  the  VHF  Direction  Finder  Operational  Test 
and  Evaluation  (OT&E)  Integration  and  OT&E  Operational  Test  Procedures  were 
executed  during  the  preliminary  OT&E  test  series,  while  only  flight  scenarios  //I 
and  y/2  were  exercised  during  the  formal  OT&E.  An  addition  to  scenario  y/4  was 
instituted  during  preliminary  OT&E  testing  to  allow  comparison  of  VDF  operator 
reported  position  with  Automated  Radar  Terminal  System  (ARTS)  IIA  reports,  which 
was  accomplished  successfully.  Flight  scenario  y/5  utilized  two  aircraft,  the  FAA 
Technical  Center  Aero  Commander  680E  and  a  light  rental  aircraft,  a  Maul  XMT-180. 
This  scenario  evaluated  the  Air  Traffic  Control  Specialist's  (ATCS)  ability  to 
handle  two  aircraft  and  the  capability  of  the  system  and  procedures  in  this 
situation.  There  were  no  operationally  significant  differences  in  bearings 
displayed  by  each  IDCU. 

The  VDF  operators  successfully  demonstrated  the  ability  to  direct  the  aircraft 
over  a  DF  antenna,  report  station  passage,  and  provide  pertinent  information 
about  aircraft  position,  landmarks,  NAVAIDS,  and  airports.  Operators  utilized 
DF  approach  procedures  to  direct  the  aircraft  to  designated  airport  runways.  The 
need  for  operator  interpretation  of  presented  data  was  noted,  and  was  generally 
successfully  accomplished  by  the  operators.  DF  approaches  were  made  to  Green  Bay, 
Marquette,  and  Eau  Claire  airports  during  preliminary  OT&E,  and  to  Stevens  Point 
and  Eau  Claire  during  formal  OT&E,  each  time  by  a  different  operator.  All 
approaches  resulted  in  an  aircraft  position  from  which  a  safe  landing  could  have 
been  made.  The  VDF  system  responded  satisfactorily  to  operator  commands  and 
provided  the  necessary  information  to  orient  the  aircraft  safely  to  an  airport. 

Table  5.3-1  sununarizes  the  DF  System  bearing  accuracy  observed  during  the  flight 
testing  conducted  from  April  1  through  7,  and  June  21  and  22,  1993.  Table  5.3-1 
is  derived  from  the  data  in  section  5  of  VHF  Direction  Finder  (VDF)  Operational 
Test  and  Evaluation  (OT&E)  Integration  and  OT&E  Operational  Test  Logs  and  Data, 
of  which  the  original  data  is  a  combination  of  angle  from  site  as  reported  by  the 
DF  system  and  angle  from  site  as  computed  from  site  and  aircraft  position.  The 
table  depicts,  in  integral  degrees,  the  mean  and  standard  deviation  of  the  error 
observed  in  orbit  and  operational  scenarios.  Also  shown  are  the  extreme  values 
less  than  (-)  or  greater  than  (+)  the  actual  bearing.  Cases  where  average  and 
standard  deviation  information  is  meaningless  are  marked  with  n/a. 

Orbital  data  was  analyzed  with  respect  to  quantifying  the  overall  error  to  be 
expected  from  each  antenna  site.  Figures  5.3-1  through  5.3-3  depict  the  azimuth 
error  at  Green  Bay,  Rhinelander,  and  Stevens  Point  as  a  function  of  distance  from 
the  site.  There  is  not  an  obvious  difference  in  the  spread  of  error  by  distance 
when  the  distance  is  40  nml  or  less.  In  the  case  of  Marquette  only  one  distance 
was  flown,  so  no  distance  comparison  was  performed.  Figures  5.3-4  through  5.3-7 
depict  the  angular  error  as  a  function  of  true  azimuth  from  antenna  site  to 
aircraft.  There  are  significant  differences  in  both  average  error  and  distribution 
of  the  error  for  FA-9964  versus  the  new  FA- 10121  antenna  sites.  Table  5.3-1  alone 
does  not  highlight  these  obvious  differences. 
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TABLE  5.3-1.  SUMMARY  OF  DF  SYSTEM  BEARING  ACCURACY 


Extremes,  Mean  Error  and  Standard  Deviation  columns  are  in  decrees 
xmt  Aircraft  Radio  Transmission  Time. 

NM  Nautical  Miles 


TABLE  5.3-1.  SUMMARY  OF  DF  SYSTEM  BEARING  ACCURACY  (Continued) 
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FIGURE  5.3-1.  GREEN  BAY  ERROR  AS  A  FUNCTION  OF  DISTANCE 
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FIGURE  5.3-2.  RHINELANDER  ERROR  AS  A  FUNCTION  OF  DISTANCE 
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FIGURE  5.3-3.  STEVENS  POINT  ERROR  AS  A  FUNCTION  OF  DISTANCE 
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FIGURE  5.3-6.  RHINELANDER  ERROR  AS  A  1  ^NOTION  OF  TRUE  AZIMUTH 


It  can  be  seen  from  section  5  of  VHF  Direction  Finder  (VDF)  Operational  Test  and 
Evaluation  (OT&E)  Integration  and  OT&E  Operational  Test  Logs  and  Data,  table  5.3-1 
and  figures  5.3-4  through  5.3-7,  that  the  FA- 9964  antennas  have  lower  average  error 
and  lower  deviations  from  average  than  the  newer  FA- 10121s.  However,  these  figures 
do  not  take  into  consideration  the  different  sites  of  the  respective  antennas.  It 
is  plausible  that  the  older  FA- 9964  antennas  are  located  at  more  favorable  sites  or 
that  installation  of  FA-10121  antenna  sites  have  different  siting  constraints  than 
the  FA- 9964  systems;  therefore,  the  FAA  would  need  to  identify  and  correct  our 
siting  procedures .  It  should  also  be  noted  that  the  Green  Bay  and  Marquette 
FA- 10121  antenna  errors  were  heavily  biased  toward  the  positive  with  both  having 
an  average  error  of  +4  degrees.  If  the  errors  were  balanced  during  commissioning 
flight  inspection,  accuracy  figures  would  improve.  However,  the  distribution  of 
errors  would  remain  unchanged  regardless  of  average  error.  Note  that  the  FA- 10121 
antenna  sites  were  prepared  for  operation  using  surveyed  target  bearings  and  that 
the  operational  rule  for  current  commissioned  DF  systems  is  that  flight  inspection 
provides  the  final  authority  for  course  alignment. 

General  azimuth  accuracy  of  DF  system- generated  positions  was  found  to  be 
independent  of  distance  within  the  40  nmi  operational  radius.  Beyond  40  nmi, 
the  error  of  the  FA-9964  sites  becomes  unpredictable,  therefore,  a  reported 
azimuth  could  be  good  or  unusable.  The  FA-10121  sites  appear  to  provide  useable 
data  beyond  40  nmi,  which  is  possibly  a  result  of  a  better  algorithm  for  discarding 
data  when  conflicting  signals  arrive  at  the  antenna  site.  It  should  be  noted  that 
atmospheric  conditions,  as  well  as  site  conditions  can  bias  these  long-range 
results . 

It  was  also  found  that  transmissions  of  approximately  1  second  in  duration  were 
less  reliable  at  the  Stevens  Point  FA-9964  site  than  at  the  Green  Bay  FA-10121 
site.  A  look  at  the  data  shows  that  a  single  point  biases  the  results,  however  the 
point  is  within  expected  variation  and  so  there  is  no  reason  to  discard  it.  At  the 
Stevens  Point  FA-9964  site,  all  data  taken  at  the  2-second  and  greater  transmission 
durations  showed  azimuth  errors  similar  between  sets.  Data  from  the  1-second 
transmission  duration  tests  was  not  Included  in  figures  5.3-1  to  5.3-7  data  in 
order  to  prevent  a  slight  biasing  of  the  results. 

The  VDF  was  operated  in  both  automatic  and  manual  mode  during  scenario  execution. 
Most  ATCSs  appeared  to  prefer  the  automatic  mode.  It  was  observed  that  the  system 
placed  a  new  aircraft  position  when  an  ATCS  communicated  with  an  aircraft  while  in 
manual  mode,  even  though  key  line  inhibit  was  activated.  The  key  line  inhibit 
design  tested  during  OT&E  does  not  meet  the  intent  of  the  PD. 

Comments  and  recommendations  for  system  operating  procedures  and  system  operating 
features  were  received  from  the  ATCSs  and  were  provided  to  the  FAA  Headquarters  AT 
representatives.  For  example,  sudden  changes  in  aircraft  position  were  observed  at 
times .  These  sudden  changes  take  the  form  of  a  leap  of  several  miles  between  two 
transmissions  closely  spaced  in  time.  However,  this  was  not  considered  an 
operational  problem  by  the  ATCSs  since  the  displacements  were  obviously  inaccurate. 
Sudden  changes  are  usually  the  result  of  the  system  placing  an  aircraft  using 
bearings  from  an  antenna  well  beyond  40  nmi.  In  this  situation,  the  ATCS  requests 
another  transmission  from  the  aircraft  to  update  the  IDCU  display. 
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Several  unsuccessful  attempts  were  made  to  obtain  reliable  strobe  Information  from 
the  VDF  from  an  aircraft  Emergency  Locator  Transmitter  (ELT)  transmitting  on  121.5 
MHz.  The  aircraft  was  placed  at  various  locations  at  the  Green  Bay  Airport.  In 
most  circumstances,  the  aircraft  was  In  line  of  sight  with  the  Green  Bay  VDF 
antenna  which  was  tuned  to  121.5  MHz.  However,  the  VDF  was  unable  to  generate  a 
reliable  strobe  from  the  ELT.  In  all  cases,  the  ELT  warbling  tone  was  audible 
through  the  DF  antenna  at  the  IDCU  position. 

Two  theories  for  the  Inability  of  the  VDF  system  to  locate  the  ELT  beacon  were 
suggested: 

a.  Several  airport  buildings  would  act  as  reflectors  for  the  VHF  signal. 

The  line  of  site  transmission  would  In  any  case  provide  a  relatively  clear  signal 
at  the  antenna.  Any  reflected  signal  Incident  on  the  antenna  should  cause  a 
multiple  signal  detect  (MSD)  condition.  The  VDF  system  Is  designed  to  Identify 
MSD  at  the  antenna.  Contrary  to  theory  1,  the  system  gave  conflicting  azimuth 
Information  rather  than  MSD  Indication.  At  the  operator  console,  an  MSD  Is 
depicted  by  " — "  In  place  of  an  azimuth  ntunber.  Some  of  the  aircraft  positions 
gave  the  MSD  Indication,  but  not  all  of  them. 

b.  The  warbling  tone  of  the  audio  Is  Incompatible  with  the  antenna  site 
hardware . 

The  antenna  site  hardware  does  make  use  of  certain  audio  frequencies  In  Its 
processing  of  an  azimuth.  This  theory  will  require  considerable  effort  to  prove 
one  way  or  the  other.  However,  the  presence  of  azimuths,  even  if  conflicting, 
argues  against  the  theory. 

The  failure  to  place  ELT  transmissions  remains  unexplained. 


6.  CONCLUSIONS. 


The  Very  High  Frequency  (VHF)  Direction  Finder  (VDF)  system  is  operationally 
satisfactory  In  terms  of  mission  effectiveness.  Some  deficiencies  exist  In  the 
maintenance  and  operational  areas,  and  deviations  from  volumes  I  and  III  of  the 
National  Airspace  System  (NAS)  System  Specification,  NAS -SS- 1000,  were  also  noted. 
Noted  deficiencies  are: 

a.  The  VDF  Technical  Instruction  (TI)  manual  and  associated  documentation 
do  not  reflect  the  current  system  configuration. 

b.  The  VDF  fails  to  meet  NAS-SS-1000,  volume  III,  paragraphs  3. 2. 1.3. 7. 2. 5 
(status  response)  and  3. 2. 1.3. 7. 2. 6  (display  response). 

c.  The  VDF  falls  to  meet  NAS-SS-1000,  volume  I,  paragraph  3.2.1.2.9(c) 
(maintenance  and  operations  support  performance  characteristics) . 

d.  The  VDF  keyline  closure  equipment  Is  not  finalized. 
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e.  An  Insufficient  number  of  Input/Output  Terminal  Type  3  (IOT-3)  computers 
to  support  effective  site  maintenance. 

f.  The  VDF  lacks  both  a  current  software  audit  and  a  fully  developed  plan 
for  software  maintenance.  No  current  methodology  for  regression  testing  future  VDF 
software  releases. 

g.  Absence  of  a  fully  developed  plan  for  map  verification. 

h.  The  VDF  lacks  finalization  of  a  plan  for  accommodating  the  56>day  map 
update  period. 

i.  The  VDF  lacks  a  methodology  for  certifying  Direction  Finder  (DF)  antenna 
alignment  and  for  verification  of  antenna  Installation. 

j .  Absence  of  surge  protection  for  the  RMMCs  and  IDCUs  according  to  the 
requirements  of  the  Purchase  Description  (PD) . 

While  Information  Display  Control  Unit  (IDCU)  and  associated  keyboard  functions  are 
satisfactory  from  an  operator  interface  aspect,  requests  for  changes  were  made  by 
the  Green  Bay  Automated  Flight  Service  Station  (AFSS)  ATCSs.  Most  ATCSs  considered 
the  FA- 10121  VDF  to  be  an  Improvement  over  the  currently  fielded  FA- 9964  and 
FA- 5530  DFs.  The  inability  of  the  VDF  to  locate  an  aircraft  Emergency  Locator 
Transmitter  (ELT)  signal  is  not  seen  as  an  operationally  significant  issue  by  AT 
representatives . 

Training  of  ATCSs  appeared  to  be  satisfactory.  However,  the  complexity  of  the  VDF 
as  compared  to  previous  generation  DFs,  requires  additional  emphasis  on  proficiency 
training.  Development  of  the  VDF  simulator  trainer  equipment  to  support  this 
increased  training  requirement  is  currently  well  underway. 


7.  RECOMMENDATIONS. 


Deployment  is  recommended  even  though  some  deficiencies  presently  exist. 

Institution  of  a  plan  of  action  to  address  the  deficiencies  is  recommended. 

The  following  recommendations  are  noteworthy: 

a.  Conduct  regression  test  of  Very  High  Frequency  (VHF)  Direction  Finder 
(VDF)  software  revisions. 

b.  Closely  scrutinize  FA- 10121  Direction  Finder  (DF)  site  maintenance  during 
initial  fielding  efforts. 

c.  Consider  balancing  observed  antenna  bearing  errors  during  commissioning 
flight  inspections.  A  detailed  airborne  analysis  of  the  site  error  by  azimuth 
should  be  conducted  during  the  first  installations,  starting  with  the  three  sites 
already  installed.  Any  special  case  problems  should  be  identified  and  corrected. 
Changes  to  site  selection  procedure  and  installation  instructions  could  be  derived 
from  the  analysis  of  the  site  errors. 
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is  AND  ABBREVIATIONS. 


ADF 

AF 

AFSS 

AFMT 

ARTS 

AT 

CHI 

COTS 

CPU 

dfi 

dBa 

DF 

DME 

EAU 

ELT 

FAA 

GPS 

GRB 

IDCU 

IFR 

lOT 

kHz 

LAT 

LOHG 

LSE 

NDT 

MOA 

MHz 

HPS 

MQT 

HSD 

MTBF 

MTP 

MSD 

NAS 

NAVAID 

NDB 

nml 

OT&E 

PD 

PM 

RHI 

RMH 

RMMC 

STE 

TI 

TVRTM 

VDF 

VHF 

VOR 

VFR 


Automatic  Direction  Finder 
Airway  Facilities 
Automated  Flight  Service  Station 
Associate  Program  Manager  for  Test 
Automated  Radar  Terminal  System 
Air  Traffic 

Computer  Human  Interface 
Commercial  Off- the -Shelf 
Central  Processor  Unit 
decibel 

decibels  above  1  milliwatt 
Direction  Finder 
Distance  Measuring  Equipment 
Eau  Claire  VDF  Antenna  Site 
Emergency  Locator  Transmitter 
Federal  Aviation  Administration 
Global  Positioning  System 
Green  Bay  VDF  Antenna  Site 
Information  Display  Control  Unit 
Instrument  Flight  Rules 
Input/Output  Terminal 
kilohertz 
Latitude 
Longitude 

Lacrosse  VDF  Antenna  Site 
Maintenance  Data  Terminal 
Military  Operations  Area 
megahertz 

Maintenance  Processor  Sxibsystem 
Marquette  VDF  Antenna  Site 
multiple  signal  detect 
Mean  Time  Between  Failures 
Master  Test  Flan 
Multiple  Signal  Detect 
National  Airspace  System 
Navigational  Aid 
Nondirectional  Radio  Beacon 
nautical  mile 

Operational  Test  and  Evaluation 
Purchase  Description 
Preventative  Maintenance 
Rhinelander  VDF  Antenna  Site 
Remote  Maintenance  Monitoring 
Reisote  Maintenance  Monitor  Computer 
Stevens  Point  VDF  Antenna  Site 
Technical  Instruction 

Test  Verification  Requirements  Traceability  Matrix 

Very  Hl^  Frequency  (VHF)  Direction  Finder 

Very  High  Frequency 

VHF  Omnidirectional  Radio  Range 

Visxial  Flight  Rules 
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